
WHAT IS CLAIMED IS: 

1. A three dimensional modeling apparatus for generating 
three dimensional shape data of a three dimensional object, 

5 comprising: 

projection means for projecting pattern light onto said three 
dimensional object; 

image input means for inputting an image of said three 
dimensional object; 
y 10 rough shape calculation means for calculating a rough shape 

w from said image; 

2 detailed shape calculation means for calculating a detailed 

^ shape from said image; and 

^ three dimensional shape data generating means for generating 

II 15 three dimensional shape data of said three dimensional object based 
on said rough shape and said detailed shape* 

2. A three dimensional modeling apparatus for generating 
three dimensional shape data of a three dimensional object, 

20 comprising: 

projection means for projecting pattern light onto said three 
dimensional object; 

image input means for inputting an image of said three 
dimensional object; 
25 rough shape input means for inputting a rough shape of said 

three dimensional object; 

detailed shape calculation means for calculating a detailed 
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shape from said image; and 

three dimensional shape data generating means for generating 
three dimensional shape data of said three dimensional object based 
on said rough shape and said detailed shape. 

3. A three dimensional modeling apparatus for generating 
three dimensional shape data of a three dimensional object, 
comprising : 

image input means for inputting an image of said three 
dimensional object, said image being obtained by projecting 
pattern light onto said three dimensional object; 

rough shape calculation means for calculating a rough shape 
from said image; 

detailed shape calculation means for calculating a detailed 
shape from said image; and 

three dimensional shape data generating means for generating 
three dimensional shape data of said three dimensional object based 
on said rough shape and said detailed shape. 

4. A three dimensional modeling apparatus for generating 
three dimensional shape data of a three dimensional object, 
comprising: 

image input means for inputting an image of said three 
dimensional object, said image being obtained by projecting 
pattern light onto said three dimensional object; 

rough shape input means for inputting a rough shape of said 
three dimensional object; 
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detailed shape calculation means for calculating a detailed 
shape from said image; and 

three dimensional shape data generating means for generating 
three dimensional shape data of said three dimensional object based 
5 on said rough shape and said detailed shape. 

5 . A three dimensional modeling apparatus according to claim 

1/ 

wherein said three dimensional shape data generating means 
10 determines the final three dimensional shape data based on the 
following rule; if the detailed shape exits inside the rough shape, 
then the detailed shape is the final three dimensional shape data 
of said object, otherwise the rough shape is taken as the final 
three dimensional shape data of said object. 

15 

6 . A three dimensional modeling apparatus according to claim 

1, 

wherein said three dimensional shape data generating means 
obtains a region where the object surface can exist based on the 
20 rough shape, to generate three dimensional shape data of said three 
dimensional object based on the rough shape, the detailed shape, 
and the region where the object surface can exist. 

7 . A three dimensional modeling apparatus according to claim 

25 1, 

wherein said detailed shape calculation means obtains a 
plurality of object surface location candidates from the input 
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image . 



8 . A three dimensional modeling apparatus according to claim 

1, 

5 wherein said detailed shape calculation means sets a region 

where the object surface cannot exist in a portion inside the rough 
shape and determines a region inside the rough shape other than 
said region where the object surface cannot exist as a region where 
the object surface can exist, to generate three dimensional shape 

0 10 data of said three dimensional object based on the rough shape, 
OS the detailed shape, and the region where the object surface can 
Nf exist . 

Ms. 

9 . A three dimensional modeling apparatus according to claim 

1 15 1, 

^ wherein said three dimensional shape data generating means 

^ determines a final object surface location candidate from a 

plurality of object surface location candidates based on a region 
where the object surface can exist and generates three dimensional 
20 shape data of said three dimensional object based on said final 
object surface location candidate. 

10. A three dimensional modeling apparatus according to 
claim 1, 

25 wherein the rough shape is calculated by said rough shape 

calculation means based on silhouette information obtained from 
the object image input by said image input means. 
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11. A three dimensional shape data recording apparatus for 
recording three dimensional shape data of a three dimensional 
object , comprising : 

three dimensional shape data generation means for generating 
5 three dimensional shape data of said three dimensional object; 

three dimensional shape data reliability assigning means for 
assigning a data reliability evaluation value to each of said three 
dimensional shape data; and 

three dimensional shape data recording means for recording 
10 three dimensional shape data to which said reliability evaluation 
value is assigned in a medium. 

12 . A three dimensional shape data recording apparatus for 
recording three dimensional shape data of a three dimensional 
15 object, comprising: 

three dimensional shape data capturing means for capturing 
three dimensional shape data of said three dimensional object; 

three dimensional shape data reliability assigning means for 
assigning a data reliability evaluation value to each of said three 
20 dimensional shape data; and 

three dimensional shape data recording means for recording 
three dimensional shape data to which said reliability evaluation 
value is assigned in a medium. 

25 13 . A three dimensional modeling method for generating three 

dimensional shape data of a three dimensional object , comprising 
the steps of: 
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projecting pattern light onto said three dimensional object; 
inputting an image of said three dimensional object; 
calculating a rough shape from said image; 
calculating a detailed shape from said image; and 
5 generating three dimensional shape data of said three 

dimensional object based on said rough shape and said detailed 
shape . 



14 . A three dimensional modeling method for generating three 

Jj§ 10 dimensional shape data of a three dimensional object, comprising 

13. 

jP the steps of : 

projecting pattern light onto said three dimensional object; 
L, inputting an image of said three dimensional object; 

~ capturing a rough shape of said three dimensional object; 

i| 15 calculating a detailed shape from said image; and 

CI generating three dimensional shape data of said three 

^ dimensional object based on said rough shape and said detailed 

shape. 

20 15 . A three dimensional modeling method for generating three 

dimensional shape data of a three dimensional object, comprising 
the steps of : 

inputting an image of said three dimensional object, said 
image being obtained by projecting pattern light onto said three 
25 dimensional object; 

calculating a rough shape from said image; 
calculating a detailed shape from said image; and 
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generating three dimensional shape data of said three 
dimensional object based on said rough shape and said detailed 
shape . 

16 . A three dimensional modeling method for generating three 
dimensional shape data of a three dimensional object, comprising 
the steps of: 

inputting an image of said three dimensional object, said 
image being obtained by projecting pattern light onto said three 
dimensional object; 

capturing a rough shape of said three dimensional object; 

calculating a detailed shape from said image; and 

generating three dimensional shape data of said three 
dimensional object based on said rough shape and said detailed 
shape . 

17. A three dimensional modeling method according to claim 

13, 

wherein said step of generating three dimensional shape data 
includes determining the final three dimensional shape data based 
on the following rule; if the detailed shape exits inside the rough 
shape, then the detailed shape is the final three dimensional shape 
data of said object, otherwise the rough shape is taken as the final 
three dimensional shape data of said object. 

18. A three dimensional modeling method according to claim 

13, 
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wherein said step of generating three dimensional shape data 
includes obtaining a region where the object surface can exist 
based on the rough shape, to generate three dimensional shape data 
of said three dimensional object based on the rough shape, the 
5 detailed shape, and the region where the object surface can exist. 

19. A three dimensional modeling method according to claim 

13, 

wherein said step of calculating detailed shape includes 
qIO obtaining a plurality of object surface location candidates from 
2 the input image . 

{T 20. A three dimensional modeling method according to claim 

safe- 

; 13, 

njlS wherein said step of calculating detailed shape includes 

r? setting a region where the object surface cannot exist in a portion 
inside the rough shape and determining a region inside the rough 
shape other than said region where the object surface cannot exist 
as a region where the object surface can exist, to generate three 
20 dimensional shape data of said three dimensional object based on 
the rough shape, the detailed shape, and the region where the object 
surface can exist. 

21. A three dimensional modeling method according to claim 

25 13, 

wherein said step of generating three dimensional shape data 
includes determining a final object surface location candidate 
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from a plurality of object surface location candidates based on 
a region where the object surface can exist and generating three 
dimensional shape data of said three dimensional object based on 
said final object surface location candidate. 

5 

22. A three dimensional modeling method according to claim 

13, 

wherein the rough shape is calculated by said rough shape 
calculation means based on silhouette information obtained from 
10 the object image input by said step of inputting an image. 

23. A three dimensional shape data recording method for 
recording three dimensional shape data of a three dimensional 
object, comprising the steps of: 

15 generating three dimensional shape data of said three 

dimensional ob j ect ; 

assigning a data reliability evaluation value to each of said 
three dimensional shape data; and 

recording three dimensional shape data to which said 
20 reliability evaluation value is assigned in a medium. 

24. A three dimensional shape data recording method for 
recording three dimensional shape data of a three dimensional 
object, comprising the steps of: 

25 capturing three dimensional shape data of said three 

dimensional object; 

assigning a data reliability evaluation value to each of said 
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three dimensional shape data; and 

recording three dimensional shape data to which said 
reliability evaluation value is assigned in a medium. 

5 25. A medium having a program recorded therein for causing 

a computer to generate three dimensional shape data of a three 
dimensional object, said program comprising the steps of: 

controlling pattern light projection section connected to 
said computer for projecting pattern light onto said three 
y 10 dimensional object; 

-~z5 inputting an image of said three dimensional object; 

2f calculating a rough shape from said image; 

^ calculating a detailed shape from said image; and 

^ generating three dimensional shape data of said three 

H 15 dimensional object based on said rough shape and said detailed 

^ shape . 

26. A medium having a program recorded therein for causing 
a computer to generate three dimensional shape data of a three 
20 dimensional object , said program comprising the steps of: 

controlling a pattern light projection section connected to 
said computer for projecting pattern light onto said three 
dimensional object; 

inputting an image of said three dimensional object; 
25 capturing a rough shape of said three dimensional object; 

calculating a detailed shape from said image; and 

generating three dimensional shape data of said three 
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dimensional object based on said rough shape and said detailed 
shape . 

27. A medium having a program recorded therein for causing 
5 a computer to generate three dimensional shape data of a three 
dimensional object, said program comprising the steps of: 

inputting an image of said three dimensional object, which 
is obtained by projecting pattern light onto said three dimensional 
object; 

O 

10 calculating a rough shape from said image; 

jf calculating a detailed shape from said image; and 

r4 generating three dimensional shape data of said three 

^ dimensional object based on said rough shape and said detailed 

l s . shape, 
U 15 

IT 28. A medium having a program recorded therein for causing 

a computer to generate three dimensional shape data of a three 
dimensional object, said program comprising the steps of: 

inputting an image of said three dimensional object, which 
20 is obtained by projecting pattern light onto said three dimensional 
object; 

capturing a rough shape of said three dimensional object; 
calculating a detailed shape from said image; and 
generating three dimensional shape data of said three 
25 dimensional object based on said rough shape and said detailed 
shape. 
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29. A medium having a three dimensional modeling program 
recorded therein according to claim 25 / 

wherein said step of generating three dimensional shape data 
includes determining the final three dimensional shape data based 
5 on the following rule; if the detailed shape exists inside the rough 
shape, then the detailed shape is the final three dimensional shape 
data of said object, otherwise the rough shape is taken as the final 
three dimensional shape data of said object. 

yg 10 30. A medium having a three dimensional modeling program 

jj* recorded therein according to claim 25, 

q : wherein said step of generating three dimensional shape data 

m includes obtaining a region where the object surface can exist 

i~j based on the rough shape, to generate three dimensional shape data 
15 of said three dimensional object based on the rough shape, the 

ST detailed shape, and the region where the object surface can exist. 

31. A medium having a three dimensional modeling program 
recorded therein according to claim 25, 

20 wherein said step of calculating a detailed shape includes 

obtaining a plurality of object surface location candidates from 
the input image . 

32. A medium having a three dimensional modeling program 
25 recorded therein according to claim 25, 

wherein said step of calculating a detailed shape includes 
setting a region where the object surface cannot exist in a portion 
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inside the rough shape and determining a region inside the rough 
shape other than said region where the object surface cannot exist 
as a region where the object surface can exist, to generate three 
dimensional shape data of said three dimensional object based on 
the rough shape, the detailed shape, and the region where the object 
surface can exist. 

33. A medium having a three dimensional modeling program 
recorded therein according to claim 25, 

wherein said step of generating three dimensional shape data 
includes determining a final object surface location candidate 
from a plurality of object surface location candidates based on 
a region where the object surface can exist and generating three 
dimensional shape data of said three dimensional object based on 
said final object surface location candidate. 

34. A medium having a three dimensional modeling program 
recorded therein according to claim 25, 

wherein the rough shape is calculated by said step of 
calculating a rough shape based on silhouette information obtained 
from the object image input by said step of inputting an image. 

35. A medium having a program recorded therein for causing 
a computer to record three dimensional shape data of a three 
dimensional object in the medium, said program comprising the steps 
of: 

generating three dimensional shape data of said three 
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dimensional object; 

assigning a data reliability evaluation value to each of said 
three dimensional shape data; and 

recording three dimensional shape data to which said 
5 reliability evaluation value is assigned in the medium. 

36. A medium having a program recorded therein for causing 
a computer to record three dimensional shape data of a three 
dimensional object in the medium, said program comprising the steps 

3 10 of: 

JE capturing three dimensional shape data of said three 

q dimensional object; 

yL assigning a data reliability evaluation value to each of said 

s 

23 three dimensional shape data; and 

j5| 15 recording three dimensional shape data to which said 

J-, reliability evaluation value is assigned in the medium. 

37. A medium having three dimensional shape data of a three 
dimensional object recorded therein, said medium holding data 

20 reliability evaluation value information for each of said three 
dimensional shape data. 

38. A three dimensional modeling apparatus for generating 
three dimensional shape data of a three dimensional object, 

25 comprising: 

projection means for projecting pattern light onto said three 
dimensional object; 
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image input means for inputting an image of said three 
dimensional object; and 

three dimensional shape calculation means for calculating a 
three dimensional shape of said three dimensional object from the 
5 input image , 

wherein said three dimensional shape calculation means 
extracts a pattern projected region and a pattern border region 
in said input image to calculate the three dimensional shape of 
said three dimensional object based on these regions. 

10 

39. A three dimensional modeling apparatus according to 
I claim 1, 

J wherein said detailed shape calculation means extracts a 

j pattern projected region and a pattern border region in said input 
JL5 image to calculate the detailed shape of said three dimensional 
[j- object based on these regions. 

40. A three dimensional modeling apparatus according to 
claim 38 , 

20 wherein said pattern light has a plurality of binary patterns . 

41. A three dimensional modeling apparatus according to 
claim 38 , 

wherein said image input means performs image input from at 
25 least two positions. 

42. A three dimensional modeling apparatus for generating 
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three dimensional shape data of a three dimensional object, 
comprising : 

projection means for projecting pattern light onto said three 
dimensional object; 
5 image input means for inputting an image of said three 

dimensional object; and 

three dimensional shape calculation means for calculating a 
three dimensional shape of said three dimensional object from the 
input image, 

ID wherein said image input means performs image input from at 

~f least two positions, 

fz_ said three dimensional shape calculation means extracts a 

M portion in said input image where color or density changes, and 
^ matching of portions where color or density changes is 

tS performed between input images, to calculate the three dimensional 
Q shape of said three dimensional object. 

43. A three dimensional modeling apparatus according to 
claim 1, 

20 wherein said image input means performs image input from at 

least two points at different locations, 

said detailed shape calculation means extracts a portion in 
said input image where color or density changes, and 

matching of portions where color or density changes is 
25 performed between input images, to calculate the three dimensional 
shape of said three dimensional object. 
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44. A three dimensional modeling apparatus according to 
claim 3, 

wherein said image input means performs image input where the 
images are captured at least two different points at different 
5 locations, 

said detailed shape calculation means extracts a portion in 
said input image where color or density changes, and 

matching of potions where color or density changes is 
performed between input images, to calculate the three dimensional 
10 shape of said three dimensional object. 

HF 45. A three dimensional modeling apparatus according to 

P claim 42, 

N 5 wherein said pattern light for projection includes multi- 

US color patterns in which adjacent patterns have different colors 
O having a hue difference of at least 90 degree or more, or a 
Q brightness difference of 0.3 or more, when the colors are 
represented in an HSV space. 

20 46. A three dimensional modeling apparatus according to 

claim 42, 

wherein said matching of the portions where color or density 
changes is obtained based on properties of the portions where color 
or density changes, which are obtained from the input images. 

25 

47. A three dimensional modeling apparatus according to 
claim 46, 
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wherein the property of the portion where color or density 
changes is color information regarding portions located to the left 
and right of, or above and below, said portion where color or density 
changes in said input image. 

48. A three dimensional modeling apparatus according to 
claim 46 , 

wherein averaging is performed on the input images when said 
property is obtained. 

49. A three dimensional modeling apparatus according to 
claim 48, 

wherein said averaging is performed for each of divided 
regions of the input image . 

50. A three dimensional modeling apparatus according to 
claim 1 , further comprising moving means for moving said projection 
means and said image input means. 

51. A three dimensional modeling apparatus according to 
claim 50, 

wherein said moving means moves said projection means and said 
image input means based on said rough shape. 

52. A three dimensional modeling apparatus for generating 
three dimensional shape data of a three dimensional object, 
comprising : 
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at least two three dimensional shape input means for obtaining 
a three dimensional shape of said three dimensional object; and 

moving means for moving said three dimensional shape input 
means , 

5 wherein said three dimensional shape input means have 

different three dimensional input accuracy, 

53. A three dimensional modeling apparatus according to 
claim 52 , 

q 10 wherein said moving means moves said three dimensional shape 

m input means having higher three dimensional input accuracy based 

^| on three dimensional shape information obtained by said three 

dimensional shape input means having lower three dimensional input 

1 accuracy. 

S 15 

2 54 . A three dimensional modeling method for generating three 
£7 dimensional shape data of a three dimensional object, comprising 

the steps of : 

projecting pattern light onto said three dimensional object; 
20 inputting an image of said three dimensional object; and 

calculating a three dimensional shape of said three 
dimensional object from the input image, 

wherein said step of calculating the three dimensional shape 
includes extracting a pattern projected region and a pattern border 
25 region in said input image to calculate the three dimensional shape 
of said three dimensional object based on these regions. 
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55. A three dimensional modeling method according to claim 

13, 

wherein said step of calculating the detailed shape includes 
extracting a pattern projected region and a pattern border region 
5 in said input image to calculate the detailed shape of said three 
dimensional object based on these regions. 

56. A three dimensional modeling method according to claim 

54, 

10 wherein said pattern light has a plurality of binary patterns . 

57. A three dimensional modeling method according to claim 

54, 

wherein said step of inputting an image includes performing 
15 image input from at least two positions. 

58 . A three dimensional modeling method for generating three 
dimensional shape data of a three dimensional object, comprising 
the steps of: 

20 projecting pattern light onto said three dimensional object; 

inputting an image of said three dimensional object; and 
calculating a three dimensional shape of said three 
dimensional object from the input image, 

wherein said step of inputting an image includes performing 
25 image input from at least two positions, 

said step of calculating the three dimensional shape includes 
extracting a portion in said input image where color or density 
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changes, and 

matching portions where color or density changes between 
input images, to calculate the three dimensional shape of said 
three dimensional object. 

5 

59. A three dimensional modeling method according to claim 

13, 

wherein said step of inputting an image includes performing 
image input where the images are captured at least two points at 
m 10 different locations, 

;5 said step of calculating a detailed shape includes extracting 

J a portion in said input image where color or density change, and 

rf matching portions where color or density changes between 

" input images, to calculate the three dimensional shape of said 

15 three dimensional object. 

W 60. A three dimensional modeling method according to claim 

58, 

wherein said pattern light for projection includes multi- 
20 color patterns in which adjacent patterns have different colors 
having a hue difference of at least 90 degrees or more, or a 
brightness difference of 0.3 or more, when the colors are 
represented in an HSV space. 

25 61. A three dimensional modeling method according to claim 

58, 

wherein said matching of the portions where color or density 
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changes is obtained based on properties of the portions where color 
or density changes, which are obtained from the input image. 

62. A three dimensional modeling method according to claim 

61, 

wherein the property of the portion where color or density 
changes is color information regarding portions located to the left 
and right of, or above and below, said portion where color or density 
changes in said input images. 

63. A three dimensional modeling method according to claim 

61, 

wherein averaging is performed on the input images when said 
property is obtained. 

64. A three dimensional modeling method according to claim 

63, 

wherein said averaging is performed for each of divided 
regions of the input images. 

65 . A three dimensional modeling method for generating three 
dimensional shape data of a three dimensional object, comprising: 

at least two three-dimensional shape inputting steps of 
obtaining the three dimensional shape of said three dimensional 
object; and 

a step of moving a three dimensional shape input section; 
wherein said three dimensional shape inputting steps have 
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different three dimensional input accuracy. 

66. A three dimensional modeling apparatus for generating 
three dimensional shape data of a three dimensional object, 
5 comprising: 

first three dimensional shape measuring means for obtaining 
rough shape information of said three dimensional object; 

second three dimensional shape measuring means for obtaining 
detailed shape information of said three dimensional object; and 
O 10 a three dimensional shape data generating section for 

W generating the three dimensional shape data of said three 

M dimensional object based on said rough shape information and said 

H*- detailed shape information. 

1J1; 15 67. A three dimensional modeling apparatus according to 

„M: claim 66, further comprising control means for controlling the 

Mb position of said second three dimensional shape measuring means, 

said control means being controlled* based on said rough shape 
information obtained by said first three dimensional measuring 
20 means. 

68 . A three dimensional modeling method for generating three 
dimensional shape data of a three dimensional object, comprising: 

a first three dimensional shape measuring step of obtaining 
25 rough shape information of said three dimensional object; 

a second three dimensional shape measuring step of obtaining 
detailed shape information of said three dimensional object; and 
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a three dimensional shape data generating step of generating 
the three dimensional shape data of said three dimensional object 
based on said rough shape information and said detailed shape 
information. 

69. A medium having a program recorded therein for causing 
a computer to generate three dimensional shape data of a three 
dimensional object , said program comprising: 

a first three dimensional shape measuring step of obtaining 
rough shape information of said three dimensional object; 

a second three dimensional shape measuring step of obtaining 
detailed shape information of said three dimensional object; and 

a three dimensional shape data generating step of generating 
the three dimensional shape data of said three dimensional object 
based on said rough shape information and said detailed shape 
information. 
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